A Collaborative Training Platform for Peer-Based Co-Construction of Knowledge and Co-Tutoring

We propose a collaborative training platform where students collectively contribute to the co-construction of the required knowledge to produce their individual semester project. Peers feedback is implemented in order to complete trainers’ supervision with peer-training. This collaborative platform is implemented as a social network, where collaborative interactions are organized 1) to engage and stimulate students to share their resources and contributions; 2) to monitor and comment peers’ contributions. 1 Pedagogical context Our purpose in this project is to investigate how a social network based configuration can be setup and applied to stimulate and engage tutee students in the co-elaboration of knowledge [1] and the co-tutoring for practical works in a blended learning context. Social networking allows introducing a mixed training approach where tutees and tutors are jointly participating in the training process. The involvement of tutees can be expected at two levels: 1) peer-tutoring, where tutees tutor their peers and 2) peer-assessment, where tutees assess their peers. Peer-training is particularly interesting in contexts where students show heterogeneous levels of background knowledge. This is the case in introductory courses in information and communications technologies for example. More advanced tutees can then share their knowledge and pull the other trainees who can benefit from more experienced peers. Trainers can supervise and monitor the global process. They can also adjust or reorient peer-tutoring when required. We design our collaborative platform by taking into consideration the issues raised in the process-oriented design methodology in six steps exposed in [2]. The six design steps are: (1) determine the learning objectives, (2) determine the expected (changes in) interaction, (3) select the task type, (4) determine whether and how much pre-structuring is needed, (5) determine group size, and (6) determine how computer support can be applied to support group-based learning. Designing the platform requires also to clearly identify the characteristics of the tutees in the classroom: background, interests... We are targeting an introductory course in information technologies and service science for bachelor students in the faculty of social and economics science. Students are coming from all the departments of the faculty. They have very heterogeneous backgrounds, expectations and interests. Most of them are not expecting to study information technologies. ICL 2009 Proceedings Page 182 Conference ICL2009 September 23 -25, 2009 Villach, Austria 2(11) We briefly survey each of the six steps and how we have addressed them in the design of our system: 1) Determine the learning objectives: Learning objectives are the first critical element to establish in terms of closed and open skills as it will impact required interaction. In our context, beside the technological skills that tutees will acquire, we are targeting two “ideal” learning objectives for students: • Closed skills: to be able to analyze and understand the design and structure of web services. • Open skills: to practice web based technologies to develop cooperative and collaborative skills and strategies. We do not expect that tutees will acquire the same skills and the same level of competence with respect to these objectives. We are aware that the kind of trainees we are targeting are probably not used to practice learning activities in cooperation and collaboration contexts. Therefore our ambitions have to be reasonable and we want to make it possible for any trainee to be able to somehow partly succeed the exercise even if they adopt an observer behavior. To give a simple definition, we consider a trainee to be an observer as long as he/she only publishes what is explicitly required and may read others’ contributions. For some trainees, reaching this stage requires a lot of efforts. Even if they do not practice cooperation and collaboration, they contribute to the global process by publishing their results and they can follow and understand how others cooperate and collaborate and how useful it can be. Moreover, for those who reach this stage, the step to jump to a deeper engagement is quite small and simple. 2) Determine the expected interaction: Our main interaction schema is focused on feedback. It is a quite simple and widespread schema in web 2.0 services that trainees may have already experienced in their typical personal practice of information technologies. It also unifies the main interaction schemas: trainee to trainee for peer-tutoring and peer-assessment; trainer to trainee for assessment and advice. It also seems to be a schema that engages trainees to cooperate: a trainee publishes his/her contribution, the contribution is then commented by other trainees, which encourages the author to react and answer to the comment. This may initiate an iterative loop where trainees converge to a common solution or answer. 3) Select the task type: Tutees are requested a succession of open and close tasks during the whole exercise. They first start with close tasks based on real case-studies that are analyzed with a predefined form they have to fill. This stage allows establishing a common and shared corpus of knowledge among the tutees. Case-studies analyzes are then the starting point for open tasks where tutees are questioning, exchanging and commenting. They conclude with an individual synthesis where they report what they have learned from the collaboration phase. All tutees are assigned the same tasks. 4) Determine whether and how much pre-structuring is needed: As raised in [2], “collaboration sometimes develops spontaneously, but more often it does not”. Therefore, one issue consists in defining how and how much interaction has to be pre-structured in advance in order to convey tutees to collaborate. In our case, during the exercise, tutees have to go from a case-study to generalization based on an analysis of the many case-studies collected by all tutees. Therefore, each tutee has to ICL 2009 Proceedings Page 183 Conference ICL2009 September 23 -25, 2009 Villach, Austria 3(11) guarantee that he/she has correctly understood and completed the analysis of his/her case study and that the others’ analyzes are correctly completed. This situation initiates feedbacks, comments, discussions... During the whole process, tutees create a common corpus of knowledge that is resulting in the database of case studies forms. It defines a shared ground of knowledge that all tutees can exploit. We adopt a low-level of prestructuring, where tutees build their interaction processes with the interaction toolbox available in the platform with limited involvement and guidance from the trainers. Finally, participation and collaboration are integrated into the way students are graded. 5) Determine group size: We do not expect to organize groups according to size criteria. In our context, we are with large (hundreds of students) classrooms where students do not know each other. Therefore forcing them to setup fixed size groups may not be a good approach to stimulate further cooperation. However, we need to split the classroom in smaller groups in order to avoid overflowing tutees. We choose to create groups according to tutees’ interests. Therefore groups are created according to topics (web services for administration, for finance, for education...). Tutees can register to the group of their choice. To avoid unbalanced groups, a poll is first organized to estimate the most interesting topics among the tutees. It sounds an interesting strategy to bring tutees to the course by starting from their specific interests and involve them in a participatory process at the beginning of the exercise. The objective of groups is not that all members of a group must collaborate. It only structures the classroom into groups of interest where tutees can focus their attention and contribution. During the exercise period, they will also be able to create their own informal network of colleagues with whom they will closely collaborate. 6) Determine how computer support can be applied to support group-based learning Our basic choice for implementing the collaborative platform is to avoid developing a specific and specialized learning management system. We also want to keep it as general as possible so that the same approach could be applied in other context. We are driven by two principles: 1) the first one is resulting from our second learning objective: it makes sense to support this objective to show how existing and widespread systems, the ones that tutees will be the most prone to face during their personal or professional activities, can help and support collective learning and knowledge production; 2) the second one is resulting from the characteristics of the tutees population we are targeting: to reduce the technology barrier and minimize the additional workload induced by the learning of the platform, we estimate that choosing an existing type of platform, that they may have already played with, might be helpful. Finally, as our main interaction type is based on feedback, we have decided to setup the main architecture of our platform as a blog farm, where each entity (user or group of users) has its own blog space where all contributions are displayed and organized anti-chronogically reflecting the evolution and progression through time. Blogs are associated to tutees and to groups of tutees. The idea is also to give a multi-level view of the collaboration process: at the user level, at the thematic group level and at the global classroom level. Blogs will present portfolios of the activities of tutees/groups/classroom. The feedback schema is implemented through the post/comments process. Users groups are integrated into the platform to subdivide the whole classroom into thematic sub-groups. The creation and management of formal sub-groups is restrained to trainers. Tutees groups are associated with their own blog space, which gathers the contributions of all the group members. ICL 2009 Proceedings Page 184 Conference ICL2009 September 23 -25, 2009 Villach, Austria 4(11) This basic architecture has to be completed to achieve the design of the platform. The main access entry to content in a blog is chronology, which is useful to follow the evolution of a process through time but is to restricted to support users to a full and flexible access. We need to complete with other entry accesses, such as search engine and tags/folksonomy. We also need to provide users with ways to keep track of the contents they are interested in with tools such as bookmarks, content tracking and monitoring systems so that they can simply access contributions and observe their evolution. Apart from users groups, we also plan to integrate some social tools such as friends-like management tools so that tutees can create and manage their own informal group of colleagues. The co-elaboration of knowledge by tutees is achieved through collaborative interactions [3], where student tutees can publish their views/ ideas/ findings/ references that all other student tutees can use for their own practice and comment, which initiates discussions inside the tutees community. All the contributions are accumulated in a common place, which tutees can search and browse. One of the most important aspects consists in initiating the knowledge co-construction process. This is particularly critical as it will impact how collective knowledge will converge [4]. For this purpose, student tutees have to start by analyzing case studies by themselves. To establish a shared and mutual initial background for the whole community we propose a common commented survey that all student tutees are requested to fill and submit for the case study they choose. The whole process is organized into the following steps: 1) tutees have to find at least one case study relevant to a theme which they select in a given list, 2) each trainee analyzes his/her case study by filling the survey form, 3) peers are checking and assessing the submitted survey forms, 4) peers ask questions, suggest improvements or corrections... by commenting the submitted forms, 5) the knowledge co-elaboration is increased by publishing related references, 6) each student trainee submit his/her personal report based on the shared knowledge accumulated during the previous steps. During these steps, trainers are constantly monitoring the activity and progress and they can intercede in the discussions and debates between the tutees whenever required. The possible interactions between the users are trainee to trainee, trainer to trainee and trainee to trainer. 2 Implementation A social network platform has been developed at the University of Geneva in order to implement the pedagogical approach described above in the context of an introductory course in information and services technology. This course is open to all the students in first year of bachelor from all the departments of the faculty of economy and social sciences. During this course, students are assigned an individual semester project where they have to analyze the situation of e-services in different areas of social and economic fields such as commerce, finance, culture... The platform has been built with Drupal [5], [6], a PHP/MySQL based content management framework. Among all available existing frameworks, Drupal has been selected because it ICL 2009 Proceedings Page 185 Conference ICL2009 September 23 -25, 2009 Villach, Austria 5(11) provides a very flexible and modular structure: Drupal is organized around a core engine providing basic features that can be extended with ad-hoc modules. In the platform, the whole students’ class is organized into groups and sub-groups in order to gather the knowledge and present it at the different appropriate levels: global for the whole class (what concerns e-services in general), thematic groups (what concerns e-service in a specific area). Sub-groups are also important to keep the number of participants involved in the collaboration to a reasonable amount to avoid overflowing tutees. Whatever is published at the global (group) level is propagated into the lower levels (thematic sub-groups). The propagation is not possible from low to high level (what concerns a specific topic may not be of global interest). Each user is assigned an individual blog. By default, whatever is published by any user is published in his/her blog. Each blog allows giving a view of the individual contribution and progress of the owner. The user can decide in which group(s) each contribution is assigned to. All published contributions are made public and available to any other users. Content types are limited to four: blog posts, forum posts, bibliographic references and forms. Each content type can be commented. Comments represent the main artifact used for establishing collaborative interactions by discussing and assessing contents between peers. Apart from administrative operations, trainers are basically provided with the same features as tutees. The main difference is in the types of content they can submit. Some specific ones are available such as announcements with the specific goal of informing trainees. Trainers’ contributions are made public as tutees' ones. One restriction is the ability to apply annotations to tutees’ content. Annotations are specific comments that can only be viewed by their author and the annotated post’s author. 2.1 Reward structure In [7], the authors remind the importance of reward structures: “Simply assigning two learners to work together independently, or under the guidance of an instructor, does not necessarily mean that the experience will be successful and will lead to the outcomes [...]. For peer coaching interventions to be successful, specific reward structures need to be in place. These reward structures influence the cognitive and affective outcomes of a learning experience by creating cooperative, competitive or individualistic behaviour.” As one of our learning objectives consists in promoting and developing the acquisition of peers cooperation skills, we expect to define a cooperative reward structure, which means a reward structure that stimulates cooperative behavior and avoid competitive and individualistic ones. Before setting up the reward structure, we need to remember that the large majority of our trainee students have never been placed in a similar pedagogical scenario, where cooperation and collaboration are promoted. It is therefore difficult to motivate their engagement by simply arguing about the potential learning outcomes they may acquire from the exercise. This may appear to them during or at the end of the exercise. Therefore we have opted for a reward structure that instigates their engagement in cooperative interactions by integrating cooperation and collaboration activities into the final exercise grade. First of all the goal is not to force trainees but to encourage them to participate to the exercise. Therefore, the exercise grade only brings additional points to the global course grade. Cooperative reward is integrated at two levels: trainee’s participation to the collective process (gathering and production of the collective knowledge) and trainees’ individual application and processing of the results of the collective process (exploitation of the collective knowledge). The first level is directly estimated from the platform by logging and tracking ICL 2009 Proceedings Page 186 Conference ICL2009 September 23 -25, 2009 Villach, Austria 6(11) trainees’ activity. The second level is indirectly estimated by considering the use of collective knowledge as a criterion in the report evaluation. We do not aim at rewarding the results or the outcomes of the cooperation and collaboration, but to reward behaviors that create positive conditions for cooperation and collaboration. The objective is to estimate the global involvement of each tutee in the collaborative knowledge building process. We do not aim at evaluating the value of the content of the individual contribution, but only the contribution itself with respect to the global collaboration goal. It means that a contribution of a given type with an erroneous statement is to be estimated at the same rank as a true one of the same type. We consider that the efforts made by the tutees to produce their contributions are equivalent and that each contribution by itself demonstrates the will to participate and contribute. The main application of the evaluation is to take this aspect into consideration to determine the final grade. This evaluation is kept “invisible”, which means that it does not affect the way contents and users are displayed on the platform (it does not show a list of the “most active users” for example). This is decided in order to avoid emulating fake contributions and competition. Although students are unambiguously told about the integration of their participation in the final grade, they are not aware of the exact process and rules to establish it. As this process is expected to be semi-automatic to automatic, publicly stating the rules would make it easy to fool the system and would therefore require introducing supplementary monitoring processes to prevent potential fraud. The last aspect we expect to take into consideration is what we call “meaningful/meaningless contributions”. Although we do not want to evaluate the value of the contributions, we would like to discriminate meaningful and meaningless contributions (and consequently remove meaningless contributions from participation). Meaningless contributions are contributions that do not bring anything or bring limited addedvalue to the collective process. For example a comment limited to “I really like your post”, or “I agree with you” is considered as meaningless (although it indicates that the author has read the contribution he/she is commenting), because it does not add any value to the collective process. We have adopted two different strategies during the two experiments described in the next section. The first one is deliberately simple and straightforward. The target is mainly to investigate if and how automatic participation evaluation is possible. For this first one, any contribution of any type is weighted with the same value at any time of the exercise. The system automatically tracks contributions for each tutee. Points are not automatically assigned to the tutee. They have to be manually validated by the trainees before being cumulated to the tutees’ global account. One objective is to be able to discriminate meaningless contributions and also to be able to identify some of their properties and characteristics. The main conclusion is that the proportion of meaningless contributions is not significant with respect to the global amount of contributions. Based on the observations made during this first experiment, we have devised a second, automatic strategy. These observations are mainly aimed at defining “unfair” behaviors. Unfair behaviors are mainly behaviors that do not contribute (voluntary or not) to the collective process. Unfair behaviors should be penalized during the exercise: 1) Many tutees have submitted their contributions at the very last time, close to the deadline and almost all at the same time: these contributions are submitted too late to be of any proficient help to the community. 2) Some tutees have only submitted “unidirectional” contributions. By unidirectional, we mean that they have limited their contribution to submitting their own content to the community such as blog posts, but they did not submitted reactive contributions such ICL 2009 Proceedings Page 187 Conference ICL2009 September 23 -25, 2009 Villach, Austria 7(11) as comments (and probably did not even examine others’ contributions): the types of these contributions limit the contribution of their authors to the community. 3) Very few meaningless contributions were submitted: we can opt for considering that meaningless contributions are marginal and do not need to be tracked. The second strategy is completely automatic. Based on our third observation, we have decided that points will be automatically added to the trainee’s account for any contribution without checking it. From our first observation we have decided to take the time period into consideration for weighting the contributions. It means that a given contribution will have a higher score at the beginning of the exercise than at the end. Contribution weights are regularly decreased during the whole exercise period. Finally, to complete our strategy, we have defined different weights according to the types of contribution. For example, a comment will always score more than a post. To avoid potential competition and penalizing trainees with narrow technological skills, the marks assigned to tutees for participation is not defined proportionally according to the maximal participation among them. We define fixed thresholds for which we consider different levels of participation (from no participation to maximal participation). More complex strategies could probably be studied such as the one proposed in [8] for evaluating forum’s posts or in [9] for evaluating Wikipedia articles. More indicators can be defined in order to fine tune the trainees’ participation. However, we need to define at which level of details we have to go. It is not useful to overload the platform with logging and tracking processes to feed a complex participation model if it does not bring much to the reward structure. 3 Experiments The proposed computer-supported semester project has been experimented twice during the autumn and spring semesters in 2008-2009. The first class number was around 140 students divided into 10 sub-groups students and the second one 400 divided into 15 sub-groups. For the first experiment, we obtained around 100 analysis forms, 200 blog posts, 120 bibliographic references, 90 comments and 50 individual project reports. For the second experiment, we obtained 464 analysis forms, 1000 blog posts, 60 bibliographic references, 2200 comments and 400 individual reports. Table 1 summarizes the general activity and provides some global statistics to evaluate tutees participation and interaction. The main difference between the two experiments is the guidelines provided to the students before and during the project. No major changes have been made to the platform except for the submission of the analysis grid form. In the first experiment, grid forms are spreadsheets that are attached to blog posts whereas in the second experiment, grid forms are online web forms. In the two experiments, the project grade incorporates the participation of the student trainee to the platform activity as described in the previous section. The activity is estimated by installing a module in the platform which allows assigning pre-defined amounts of points to users according to the type of content they submit. Tutees do not have access to their current account. 3.1 Emerging cooperative and collaborative behaviors During the two experiments, we have noticed that, although sparsely, some spontaneous cooperative and collaborative behaviors have emerged. Spontaneous behaviors are behaviors that have emerged directly initiated by trainees without any guidelines or requirements from the trainers. They are proactive behaviors. From our point of view these behaviors are ICL 2009 Proceedings Page 188 Conference ICL2009 September 23 -25, 2009 Villach, Austria 8(11) indicators that demonstrate that some of the trainees have fully reached one of our objectives: develop technology enhanced cooperation and collaboration strategies to address problems. Among the different situations, we briefly describe some of the most significant and prominent situations: Case 1: A few students have collaborated to produce a set of possible questions for the final test. After the publication of the questions set a bigger community has emerged to compare and discuss solutions, but also to discuss about the questions themselves. Case 2: After the publication of a sample of the final test, some students have proposed to publish, compare and discuss their solution to the questions proposed in the test model. We can notice that these cases have usually been initiated after the proposal of one trainee on the platform. Other attempts have been made, such as a proposal in one thematic sub-group to collectively discuss and summarize about the group topic, but without meeting the interest of other trainees. Successful initiatives are the ones that directly bring immediate individual outcomes such as preparing the final test. We have also observed some implicit emerging collective behaviors, such as for example references submitted to the platform that are collectively adopted to write the final reports.